Deblocking filters are used in the video coding standards in order to combat blocking artifacts. The blocking artifacts arise because the original video frames are split into blocks which are processed relatively independently. The blocking artifacts can, for instance, arise due to different intra predictions of the blocks, quantization effects and motion compensation. Two particular variants of deblocking are described below.
In state of the art video coding, such as H.264, there is a deblocking filter, also denoted loop filter, after prediction and residual reconstruction, but before storage of the reconstruction for later reference when encoding or decoding the subsequent frames. The deblocking filtering consists of several steps such as filter decisions, filtering operations, a clipping function and changes of pixel values. The decision to filter the border or not is made based on evaluation of several conditions. Filter decisions depend on macroblock (MB) type, motion vector (MV) difference between neighboring blocks, whether neighboring blocks have coded residuals and on the local structure of the current and/or neighboring blocks.
Then the amount of filtering for a pixel depends, among others, on the position of that pixel relative to the block border or boundary and on the quantization parameter (QP) value used for residual coding.
The filter decision is based on comparing three pixel differences with thresholds. The thresholds are adapted to the quantization parameter (QP). For instance, assume a vertical block boundary of:                abcd|efghwhere a, b c and d denote the pixel values of the pixels of a row of pixels in the current block and e, f, g and h denote the corresponding pixel values of the pixels of a corresponding row of pixels in the neighboring block. If the following conditions are fulfilled the filter decision is positive, e.g. abs(d−e)<thr1, abs(c−d)<thr2, and abs(e−f)<thr2, where thr1 and thr2 are adapted based on QP.        
There are two filtering modes in H.264. In the first filtering mode, referred to as normal filtering, filtering can be described with a delta value with which filtering changes the current value. The filtering for the pixels closest to the block boundary is d′=d+delta and e′=e−delta, where delta has been clipped off to a threshold ±thr3 to a value that is constrained by the QP. More filtering is thereby allowed for high QP than for low QP. Clipping can be described as delta_clipped=max(−thr3, min(thr3, delta)), where thr3 is controlling the filter strength. A larger value of thr3 means that the filtering is stronger which means that a stronger low-pass filtering effect will happen.
The filter strength can be increased if any of the following two conditions also holds, e.g. abs(b−d)<thr2 and abs(e−g)<thr2. The filter strength is adapted by clipping the delta less, e.g. allow for more variation.
The second filtering mode, referred to as strong filtering, is applied for intra macroblock boundaries only, when the following condition is fulfilled abs(d−e)<thr1/4.
For more information of deblocking filtering in H.264 reference is made to List et al., Adaptive Deblocking Filter, IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, no. 7, July 2003.
In the draft HEVC (High Efficiency Video Coding) specification “Test Model under Consideration”, ITU-T SG16 WP3 document, JCTVC-B205, Chapter 6.5 In-loop filter process, the deblocking filter works differently from H.264. The filtering is performed if at least one of the blocks on the side of the boundary is intra, or has non-zero coefficients, or the difference between the motion vector components of the blocks is greater or equal to one integer pixel. For example, when filtering the border between the blocks with a vertical block boundary ofp3ip2ip1i,p0i|q0iq1iq2iq3i with pji denoting the pixel value of pixel number j of row number i in the current block and qji denoting the pixel value of pixel number j of row number i in the neighboring block, i=0 . . . 7, j=0 . . . 3, then the following condition should also be satisfied:d=|p22−2×p12+p02|+|q22−2×q12+q02|+|p25−2×p15+p05|+q25−2×q15+q05|<βwhere β depends on QP. In the above mentioned HEVC specification, there is a table of β, where β increases with QP.
If the conditions are fulfilled and filtering is done between the current block and the neighboring block, one of two types of filtering is performed, referred to as weak and strong filtering, respectively. The choice between the strong and the weak filtering is done separately for each line depending on the following conditions. For each line i=0 . . . 7, the strong filtering is performed if all the following conditions are true, otherwise, weak filtering is performed:d<(β>>2)(|p3i−p0i|+|q0i−q3i|)<(β>>3)|p0i−q0i|<((5×tC+1)>>1)where tC and β depend on QP and >> denotes a right shift operator.
Weak filtering is performed based on the above conditions. The actual filtering works by computing an offset (Δ), adding it to the original pixel value and clip the sum to a filtered output pixel value in the range of 0-255:Δ=Clip(−tC,tC,(13×(q0i−p0i)+4×(q1i−p1i)−5×(q2i−p2i)+16)>>5))p0i=Clip0-255(p0i+Δ)q0i=Clip0-255(q0i−Δ)p1i=Clip0-255(p1i+Δ/2)q1i=Clip0-255(q1i−Δ/2)where the clip function Clip(A, B, x) is defined as Clip(A, B, x)=A if x<A, Clip(A, B, x)=B if x>B and Clip(A, B, x)=x if A≤x≤B and Clip0-255(x) is defined as Clip(0, 255, x).
Strong filtering mode is performed by the following set of operations:p0i=Clip0-255((p2i+2×p1i+2×p0i+2×q0i+q1i+4)>>3)q0i=Clip0-255((p1i+2×p0i+2×q0i+2×q1i+q2i+4)>>3)p1i=Clip0-255((p2i+p1i+p0i+q0i+2)>>2)q1i=Clip0-255((p0i+q0i+q1i+q2i+2)>>2)p2i=Clip0-255((2×p3i+3×p2i+p1i+p0i+q0i+4)>>3)q2i=Clip0-255((p0i+q0i+q1i+3×q2i+2×q3i+4)>>3)
Deblocking filtering decisions according to HEVC can lead to inaccurate deblocking filtering over block boundaries for certain blocks. In particular, neighboring blocks having different levels of local structures could be handled incorrectly in HEVC by filtering one of the blocks too much to thereby represses and filter away local structures in the block.